-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
extensions framework + UEFI x86 + rpi4b + core changes/fixes + doc generator #3282
extensions framework + UEFI x86 + rpi4b + core changes/fixes + doc generator #3282
Conversation
2e639e9
to
2e79673
Compare
This is just a preview. If I have blood left in me, I'll still
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made one quick review and will need to do a few more to understand it well. Generally looks ok.
6096817
to
417b025
Compare
Test images are up!
|
alright trying to break things now |
A quick glance. Questions.
/# use new code If the existing code is being modified, the old code is placed after "else" construction and adding the USE_EXT_NEW_CODE variable to the config (any variable name, at your discretion, this is just an example). This will allow at any time (for example, there was an incomprehensible problem with the assembly behavior) to disable all new fragments with one variable in order to have a classic build system and exclude any influence of the new code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
information in the discussion thread
First, thanks for the review @150balbes
Indeed. x86 was an accident. The main body of code is UEFI support via GRUB for aarch64 (I developed and tested on Ampere VMs, with OVMF/Tianocore firmware), but soon realised that x86 was very easy to add and costed nothing. Right now x86 has a lot of interest, from people with laptops to servers to x86 SBCs. Maybe I should rename the x86 images "AMDbian" just for laughs. PS: See screenshots above! aarch64 UEFI stuff.
I understand your concern, but IMHO using these kind of constructs would negate the long-term benefits of having an extensibility framework. With time (and bugfixes, of course) we will trust the extensibility system. For example, try building this branch with I also plan to add strict |
417b025
to
e5a9762
Compare
v3 force pushed.
|
e5a9762
to
96a06ae
Compare
v4 force pushed.
|
96a06ae
to
054348a
Compare
v6: with many fixes. things should work with both current and edge. |
- this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
… gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
054348a
to
9afbd5c
Compare
I rebased and pushed v6 again (no other changes) |
- v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
9afbd5c
to
d9e5d84
Compare
v7 with "final" small fixes pushed. |
(many more builds here, also |
The continuation of the conversation in #3298 |
Attention: this PR was NOT merged into master, but into a different/shared branch (for more reviews). |
armbian-firmware
andlinux-firmware-raspi2
by removing the later.KERNELDIR
for these 3 new kernels, until we're sure they're safe to mix.edge
andcurrent
extensions framework (née "fragments")
extensions framework; meta-extensions: auto-docs and sample extension gen
new extension methods and features via config variables in core Armbian
SKIP_EXTERNAL_TOOLCHAINS=yes
- does not download or use any linaro toolchains, only build host-installed onesSKIP_BOOTSPLASH=yes
- does not patch kernel for splash fileEXTRA_BSP_NAME=xyz
- allows for BSP variants, useful for when extensions modify the BSPEXTRA_ROOTFS_MIB_SIZE=x
- add x mib's to rootfs size, for use with very small imagesKERNEL_EXTRA_TARGETS
- what extra targets to make kernel for, default to "modules dtbs"BOOTCONFIG=none
- does not build nor install u-boot; also doesn't handle bootscripts et alunset KERNELSOURCE
- does not build nor install kernel, nor build initrd, nor build nor install firmwareARMHF_ARCH=skip
- does not add armhf to apt/dpkg, thus pure arm64SKIP_ARMBIAN_REPO=yes
- results in armbian.list.disabled in the final imageAPT_EXTRA_DIST_PARAMS
with apt-cacher-ng options and use it forPACKAGE_LIST_INSTALL/REMOVE
et alUBUNTU_MIRROR
defaultUEFISIZE=256
to enable, mountUEFI_MOUNT_POINT=/boot/efi
, first on disk but endsup at
$uefipart
=15)BIOSSIZE=1
to enable, second on disk but ends up at partition 14)UEFI_FS_LABEL="armbiefi"
- to set the FAT label for the EFI partition, visible in Win/Macadd_host_dependencies()
extension method; fillEXTRA_BUILD_DEPS="pkg pkg2"
to install to host before toolchainsdownload
pre_prepare_partitions()
extension method, for custom partition size calculationscreate_partition_table()
extension method, used to do full-custom partitioning ifUSE_HOOK_FOR_PARTITION=yes
post_create_partitions()
extension method, mostly for easy debuggingpost_write_sdcard()
extension method, where you can also setSKIP_VERIFY=yes
to skip sdcard verificationpre_install_kernel_debs()
andpost_install_kernel_debs()
extension methods.update-initramfs
failure and abort build with useful message if it doesexit_with_error()
andunmount_on_exit()
ERROR_DEBUG_SHELL=yes
, drop into a shell before unmounting/deleting everything, so we can inspect what wentwrong
apt-get remove PACKAGE_LIST_BOARD_REMOVE
packages, so any errors while removing areeasy to understand
extensions framework; refactor tool fetching/building into extensions
sunxi-tools
extension, enabled by 2 differentsunxi
family includes ("reuse" example)marvel-tools
extension, enabled by 2 differentmvebu
family includesrkbin-tools
extension, enabled byrockship64_common
family includeamlogic-fip
/c2-blobs
stuff refactored directly intomeson64_common.inc
("single-use" example).wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension
growroot
-awareness toarmbian-resize-filesystem
armbian-resize-filesystem
does not deal correctly with the UEFI layoutgrowroot
is installed on UEFI images by default, that handles growing partition during initramfsarmbian-resize-filesystem
handlesresize2fs
only, and works.linux-generic
kernel only forcurrent
edge
now builds it's own pure-mainline5.15.y
kernel, for both x86 and arm64.config
taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et alSKIP_KERNEL_SYMLINK=yes
, tested inbuilddeb
.wip for the RaspberryPi 4B via new flash-kernel extension